Screen sharing system, screen sharing method, and storage medium

ABSTRACT

A screen sharing system includes information processing apparatuses that display the same content on their respective screens as a shared screen. Each of the information processing apparatuses includes an operation right switching unit configured to acquire an operation right, a transmission data generating unit configured to generate transmission data including operational information of an input operation, and a drawing unit configured to draw a display object on the screen. At a transmitting information processing apparatus with the operation right, the drawing unit draws the display object according to the operational information, and the transmission data generating unit generates the transmission data including the operational information and sends the transmission data to a receiving information processing apparatus without the operation right. At the receiving information processing apparatus, the drawing unit draws the display object according to the operational information in the transmission data received from the transmitting information processing apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 16/287,328, which is a continuation application ofU.S. patent application Ser. No. 16/012,031, which is a continuationapplication of U.S. patent application Ser. No. 15/067,417, which is acontinuation application of U.S. patent application Ser. No. 14/751,281,which is a continuation application of U.S. patent application Ser. No.14/257,062, which is a continuation application of U.S. patentapplication Ser. No. 13/272,283, which is based upon and claims thebenefit of priority of Japanese Patent Application No. 2010-240005 filedon Oct. 26, 2010 and Japanese Patent Application No. 2011-167715 filedon Jul. 29, 2011, the entire contents of which are incorporated hereinby reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

A certain aspect of this disclosure relates to a technology for sharinga display screen.

2. Description of the Related Art

For example, Japanese Laid-Open Patent Publication No. 2005-318589discloses a technology that makes it possible to exchange information inreal time in a multimedia conference. In the disclosed technology, aserver captures a display screen of a client terminal and provides thecaptured screen to other client terminals.

Thus, in a related-art technology, screen information is sent andreceived in real time among multiple information processing apparatuseshaving a communication function to share a display screen. Such atechnology for sharing a display screen is used, for example, in aconference where participants discuss a topic while viewing the samedocument.

With the related-art technology, however, the load of communicationsamong information processing apparatuses for sharing a display screenand the processing load of the information processing apparatuses indrawing received screens tend to become high.

With the related-art technology, image data (screen information) of adisplay screen are sent and received among information processingapparatuses. Also with the related-art technology, it is necessary tosend and receive image data frequently (at a high communicationfrequency) to improve real-time screen sharing. Thus, the related-arttechnology increases the amount of data sent and received amonginformation processing apparatuses and increases the communication andprocessing loads.

These problems may also increase the time necessary for screen sharing(e.g., communication time and screen drawing time) and thereby degradereal-time screen sharing. Apparently, it is preferable to performreal-time screen sharing by sending and receiving a smaller amount ofdata among information processing apparatuses.

SUMMARY OF THE INVENTION

In an aspect of this disclosure, there is provided a screen sharingsystem including information processing apparatuses connected to eachother via a data communication channel and configured to display thesame content on their respective screens as a shared screen. Each of theinformation processing apparatuses includes an operation right switchingunit configured to acquire an operation right for performing an inputoperation on the shared screen in response to an operation right requestentered by a user, a transmission data generating unit configured togenerate transmission data including operational information of theinput operation, and a drawing unit configured to draw a display objecton the screen. One of the information processing apparatuses that hasacquired the operation right functions as a transmitting informationprocessing apparatus and each of the other information processingapparatuses without the operation right functions as a receivinginformation processing apparatus. At the transmitting informationprocessing apparatus, the drawing unit draws the display objectaccording to the operational information of the input operationperformed by the user, and the transmission data generating unitgenerates the transmission data including the operational informationand sends the transmission data to the receiving information processingapparatus. At the receiving information processing apparatus, thedrawing unit draws the display object on the screen according to theoperational information in the transmission data received from thetransmitting information processing apparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 13 are drawings illustrating an exemplary configuration ofa screen sharing system according to an embodiment;

FIG. 2 is a block diagram illustrating an exemplary hardwareconfiguration of an information processing apparatus according to theembodiment;

FIG. 3 is a block diagram illustrating an exemplary functionalconfiguration of an information processing apparatus according to theembodiment;

FIGS. 4A and 4B are drawings illustrating exemplary transmission dataincluding operational information according to the embodiment;

FIG. 5 is a sequence chart illustrating an exemplary process ofproviding a screen sharing service according to the embodiment;

FIG. 6 is a drawing illustrating an exemplary screen of a presenter andan exemplary screen of an audience;

FIG. 7 is a sequence chart illustrating an exemplary process of sharinga screen according to a first variation;

FIG. 8 is a drawing illustrating exemplary transmission data includingoperational information according to the first variation;

FIG. 9 is a graph illustrating an exemplary relationship betweenoperation time and an adjustment factor according to the firstvariation;

FIG. 10 is a drawing used to describe an exemplary process of sharing ascreen where a pointer is moved according to the first variation;

FIG. 11 is a block diagram illustrating an exemplary functionalconfiguration of an information processing apparatus according to asecond variation;

FIG. 12 is a sequence chart illustrating an exemplary process ofswitching screen sharing modes according to the second variation;

FIGS. 13A and 13B are drawings illustrating an exemplary configurationof a screen sharing system according to a third variation; and

FIG. 14 is a block diagram illustrating an exemplary hardwareconfiguration of a projection apparatus according to the thirdvariation.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred Embodiments of the Present Invention are described below withreference to the accompanying drawings.

<System Configuration>

FIGS. 1A and 1B are drawings illustrating an exemplary configuration ofa screen sharing system 1000 according to an embodiment.

As illustrated in FIG. 1A, the screen sharing system 1000 includesinformation processing apparatuses 100 ₁-100 _(n) (may be called aninformation processing apparatus 100 or information processingapparatuses 100 when distinction is not necessary) that are connectedvia a data communication channel N such as a network (e.g., a local areanetwork (LAN)). In this embodiment, it is assumed that the informationprocessing apparatuses 100 wirelessly communicate with each other, andthe data communication channel N is implemented by a wireless LAN or aBluetooth (trademark or registered trademark) network.

The information processing apparatus 100 may include a radiocommunication function, an information input function, and aninformation display function. Also in this embodiment, it is assumedthat the information processing apparatus 100 is a portable informationterminal such as a smartphone or a tablet PC rather than a desktoppersonal computer (PC).

FIG. 1B illustrates an exemplary case where the screen sharing system1000 is used for a presentation meeting. In this exemplary case, thescreen sharing system 1000 is used as described below.

The information processing apparatuses 100 ₁-100 ₄ detect an accesspoint AP provided in a meeting room and establish connections with awireless LAN.

Each participant of the presentation meeting starts a screen sharingapplication (i.e., software that provides a screen sharing function) onone of the information processing apparatuses 100 ₁-100 ₄ and displays apresentation material (hereafter called a display content) on thescreen. As a result, the display content is shared by the informationprocessing apparatuses 100 ₁-100 ₄.

For example, the display content may be stored in a storage area of theinformation processing apparatus 100 or in a storage area of an externalapparatus such as a file server. Each participant of the presentationmeeting operates the screen sharing application on the informationprocessing apparatus 100 to access the storage area and thereby obtainthe display content.

Here, one of the participants who performs a presentation is called apresenter and other participants are called audiences A, B, and C. Thepresenter operates the screen sharing application on the informationprocessing apparatus 100 ₁ and reports to the audiences A, B, and C thatthe presenter (or the information processing apparatus 100 ₁) has aright (hereafter called an operation right) to perform input operationson a shared screen. For example, the information processing apparatus100 ₁ broadcasts a message indicating acquisition of the operation rightto the information processing apparatuses 100 ₂-100 ₄ of the audiencesA, B, and C.

When an input operation such as a page turning operation or a pointeroperation is performed by the presenter, the information processingapparatus 100 ₁ sends operational information indicating the inputoperation to the information processing apparatuses 100 ₂-100 ₄. Then,the information processing apparatuses 100 ₂-100 ₄ of the audiences A,B, and C reproduce the input operation on their screens based on theoperational information. Through the above process, a display screenreflecting an input operation is shared among the information processingapparatuses 100 of the screen sharing system 1000.

With the above configuration, the screen sharing system 1000 can providea screen sharing service for information processing apparatuses.

<Hardware Configuration>

FIG. 2 is a block diagram illustrating an exemplary hardwareconfiguration of the information processing apparatus 100 according tothe embodiment.

As illustrated in FIG. 2, the information processing apparatus 100 mayinclude an input unit 101, a display unit 102, a drive unit 103, arandom access memory (RAM) 104, a read only memory (ROM) 105, a centralprocessing unit (CPU) 106, an interface unit 107, and a hard disk drive(HDD) 108 that are connected to each other via a bus B.

The input unit 101 includes, for example, a touch panel and is used toinput instructions (or operation signals) to the information processingapparatus 100. The display unit 102 displays, for example, processingresults of the information processing apparatus 100.

The interface unit 107 connects the information processing apparatus 100to the data communication channel N. The information processingapparatus 100 can communicate with other information processingapparatuses 100 via the interface unit 107.

The HDD 108 is a non-volatile storage medium for storing variousprograms and data. For example, the HDD 108 stores basic software (e.g.,an operating system such as Windows (trademark/registered trademark) orUNIX (trademark/registered trademark)) for controlling the entireinformation processing apparatus 100, and applications (e.g., a screensharing application) that are executed on the basic software toimplement various functions. The HDD 108 may manage the stored programsand data using a file system and/or a database (DB).

The drive unit 103 is an interface between the information processingapparatus 100 and a removable storage medium 103 a. The informationprocessing apparatus 100 can read and write data from and to the storagemedium 103 a via the drive unit 103. Examples of the storage medium 103a include a secure digital (SD) memory card, a universal serial bus(USB) memory, a compact disk (CD), and a digital versatile disk (DVD).

The ROM 105 is a non-volatile semiconductor memory (storage unit) thatcan retain data even when the power is turned off. For example, the ROM105 stores programs and data such as a basic input/output system (BIOS)that is executed when the information processing apparatus 100 is turnedon, and system and network settings of the information processingapparatus 100. The RAM 104 is a volatile semiconductor memory (storageunit) for temporarily storing programs and data. The CPU 106 loadsprograms and data from storage units (e.g., the HDD 108 and the ROM 105)into the RAM 104 and executes the loaded programs to control theinformation processing apparatus 100 and to perform various functions.

With the above hardware configuration, the information processingapparatus 100 can provide various information processing functions (orservices).

<Screen Sharing Function>

An exemplary screen sharing function of this embodiment is describedbelow.

In the screen sharing system 1000 of this embodiment, the same contentis displayed on the screens of the information processing apparatuses100 ₁ through 100 _(n). The information processing apparatus 100 ₁ withthe operation right sends operational information indicating a performedinput operation to the other information processing apparatuses 100 ₂through 100 _(n). The information processing apparatuses 100 ₂ through100 _(n) receive the operational information and reproduce the inputoperation on their screens based on the received operationalinformation. The screen sharing system 1000 of this embodiment providesa screen sharing function as described above.

With the related-art technology, the load of communications amonginformation processing apparatuses performed to share a display screenand the processing load of the information processing apparatuses indrawing a received display screen (image data) tend to become high. Thisis because, with the related-art technology, image data (screeninformation) of a display screen reflecting a performed input operationare sent and received among information processing apparatuses. Such aconfiguration makes it necessary to frequently (at a high communicationfrequency) send and receive a large amount of data among the informationprocessing apparatuses.

These problems of the related-art technology may also increase the timenecessary for screen sharing (e.g., communication time and screendrawing time) and thereby degrade real-time screen sharing.

In the screen sharing system 1000 of this embodiment, to prevent orreduce the problems of the related-art technology, a display content isshared beforehand among the information processing apparatuses 100 ₁-100_(n) and only operational information indicating input operations issent and received (i.e., exchanged) among the information processingapparatuses 100 ₁-100 _(n). In other words, in this embodiment, insteadof periodically sending and receiving large-size image data, minimumdata necessary to reproduce input operations are sent and received amonginformation processing apparatuses sharing basic display data. Thisconfiguration makes it possible to reduce the amount of data sent andreceived among information processing apparatuses.

Thus, the screen sharing system 1000 of this embodiment makes itpossible to share a display screen reflecting an input operation byexchanging a small amount of data among information processingapparatuses.

An exemplary functional configuration of the information processingapparatus 100 is described below.

FIG. 3 is a block diagram illustrating an exemplary functionalconfiguration of the information processing apparatus 100 according tothe embodiment.

As illustrated in FIG. 3, the information processing apparatus 100 mayinclude a communication control unit 11, an operation right switchingunit 21, an operation type determining unit 22, a transmission datagenerating unit 23, and a drawing unit 24 as functional units. Thescreen sharing function of this embodiment is implemented throughcollaboration among the functional units of the information processingapparatuses 100.

The communication control unit 11 is a functional unit that controlscommunications with other information processing apparatuses 100. Morespecifically, the communication control unit 11 controls the interfaceunit 107, and thereby transmits and receives data to and from the otherinformation processing apparatuses 100.

The operation right switching unit 21 is a functional unit that enablestransfer of an operation right for performing an input operation on ashared screen from one information processing apparatus 100 to anotherinformation processing apparatus 100. The operation right switching unit21 displays a graphical user interface (GUI) for receiving an operationright request and acquires the operation right when the operation rightrequest is received.

More specifically, when an operation right request is received, theoperation right switching unit 21 enables a function (e.g., a touchinput function) for receiving input operations on the correspondinginformation processing apparatus 100, and reports the acquisition of theoperation right via the communication control unit 11 to the otherinformation processing apparatuses 100. At the other informationprocessing apparatuses 100, the operation right switching unit 21receives the report via the communication control unit 11 and disablesthe function for receiving input operations in response to the report.Through the above process, the operation right on the shared screen istransferred from one information processing apparatus to anotherinformation processing apparatus.

The transfer of the operation right results in a change in the directionof data communications among the information processing apparatuses 100.In the example of FIG. 3, an information processing apparatus 100 a(hereafter called a transmitting information processing apparatus) hasthe operation right and sends operational information, and aninformation processing apparatus 100 b (hereafter called a receivinginformation processing apparatus) does not have the operation right andreceives the operational information.

The operation type determining unit 22 is a functional unit thatdetermines the type of input operation. Exemplary input operations orinput operation types include a page turning operation and a pointeroperation. The operational information sent from the transmittinginformation processing apparatus 100 a varies depending on the type ofinput operation. Therefore, it is necessary to determine the type ofinput operation before generating transmission data including theoperational information. Different types of input operations input viathe input unit 101 are detected as different operation events. Theoperation type determining unit 22 receives operation events from theinput unit 101 and determines the types of input operations based on thereceived operation events.

The type of input operation may be determined by any other method. Forexample, when the information processing apparatus 100 employs asoftware configuration where different program codes (or modules) areexecuted depending on performed input operations, the operation typedetermining unit 22 may be configured to determine the type of inputoperation based on the executed program code (or module).

The transmission data generating unit 23 is a functional unit thatgenerates transmission data including operational information indicatingan input operation. The transmission data generating unit 23 generatestransmission data based on operational information corresponding to thedetermined (type of) input operation. For example, the transmission datagenerating unit 23 generates transmission data as illustrated in FIGS.4A and 4B.

FIGS. 4A and 4B are drawings illustrating exemplary transmission dataincluding operational information.

In a case where the operation type determining unit 22 determines that areceived input operation is a page turning operation, the transmissiondata generating unit 23 generates transmission data 23D₁ as illustratedin FIG. 4A. The transmission data 23D₁ include, as the operationalinformation, a page number or page identification information (page=n)of a page (a destination page) to be displayed as a result of the pageturning operation. In a case where the operation type determining unit22 determines that a received input operation is a pointer operation,the transmission data generating unit 23 generates transmission data23D₂ as illustrated in FIG. 43. The transmission data 23D₂ include, asthe operational information, coordinates or pointer position information(x=a, y=b) indicating the position of the pointer on the screen.Hereafter, “transmission data 23D” may be used as a generic term for thetransmission data 23D₁ and the transmission data 23D₂.

The transmission data generating unit 23 outputs the generatedtransmission data 23D to the communication control unit 11 and requeststhe communication control unit 11 to send the transmission data 23D.With this configuration, only the minimum data necessary to reproduce aninput operation on the screen are exchanged among the informationprocessing apparatuses 100.

The operation type determining unit 22 and the transmission datagenerating unit 23 are activated and deactivated depending on whetherthe information processing apparatus 100 has the operation right. In theexample of FIG. 3, the operation type determining unit 22 and thetransmission data generating unit 23 of the information processing 100 ahaving the operation right are activated. Meanwhile, the operation typedetermining unit 22 and the transmission data generating unit 23 of theinformation processing 100 b having no operation right are deactivated.

The drawing unit 24 is a functional unit that draws a display object onthe screen. The drawing unit 24 draws a display object according to areceived operation event or operational information in receivedtransmission data. For example, the drawing unit 24 draws a displayobject as described below.

At the transmitting information processing apparatus 100 a, the drawingunit 24 draws a display object according to a received operation event.For example, when a page turning operation is performed on thetransmitting information processing apparatus 100 a, the drawing unit 24receives the corresponding operation event and draws a page object of adestination page on the screen. Meanwhile, when a pointer operation isperformed and the corresponding operation event is received, the drawingunit 24 draws a pointer object on the screen.

At the receiving information processing apparatus 100 b, the drawingunit 24 draws a display object according to operational information intransmission data received via the communication control unit 11. Forexample, when the transmission data 23D₁ including a page number arereceived, the drawing unit 24 draws a page object of a page indicated bythe page number on the screen to switch pages. Meanwhile, when thetransmission data 23D₂ including coordinates are received, the drawingunit 24 draws a pointer object on the screen according to thecoordinates. Thus, at the receiving information processing apparatus 100b, an input operation performed at the transmitting informationprocessing apparatus 100 a is reproduced on the screen according toreceived operation information.

As described above, the screen sharing function of this embodiment isimplemented through collaboration among the functional units of theinformation processing apparatuses 100. In other words, the screensharing function of this embodiment is implemented by executing screensharing application programs (software for implementing the screensharing function) installed in the information processing apparatuses100 constituting the screen sharing system 1000. More particularly, ineach information processing apparatus 100, the screen sharingapplication program is loaded by a processing unit (e.g., the CPU 106)from a storage unit (e.g., the HDD 108 or the ROM 105) into a memory(e.g., the RAM 104) and is executed to perform a process as describedbelow and thereby implement the screen sharing function.

An exemplary process (collaboration among functional units) of providinga screen sharing service is described in more detail below.

<Process of Providing Screen Sharing Service>

FIG. 5 is a sequence chart illustrating an exemplary process ofproviding a screen sharing service according to the embodiment. In FIG.5, it is assumed that the transmitting information processing apparatus100 a has acquired the operation right and transmits operationalinformation and the receiving information processing apparatus 100 breceives the operational information.

(1) Sharing Screen Reflecting Page Turning Operation

As illustrated in FIG. 5, when a page turning operation is performed onthe screen of the transmitting information processing apparatus 100 a, acorresponding operation event is input to the drawing unit 24 and theoperation type determining unit 22.

According to the operation event, the drawing unit 24 draws a pageobject of a destination page to be displayed as a result of the pageturning operation on the screen and thereby switches pages (step S101).

Meanwhile, the operation type determining unit determines that theperformed input operation is a page turning operation based on theoperation event (step S102), and requests the transmission datagenerating unit to generate transmission data 23D₁ (step S103). In thisstep, the operation type determining unit 22 obtains the page number ofthe destination page from the operation event and sends the page numberas operational information to the transmission data generating unit 23.

The transmission data generating unit 23 generates the transmission data23D₁ including the page number (step S104), and requests thecommunication control unit 11 to send the transmission data 23D₁ (stepS105). In this step, the transmission data generating unit 23 sends thegenerated transmission data 23D₁ to the communication control unit 11.

Then, the communication control unit 11 sends the transmission data 23D₁including the page number of the destination page to the receivinginformation processing apparatus 100 b (step S106). For example, thecommunication control unit 11 of the transmitting information processingapparatus 100 a transmits the transmission data 23D₁ according tonetwork setting information (e.g., an Internet protocol (IP) address) ofthe receiving information processing apparatus 100 b which is obtainedin a broadcasting process.

At the receiving information processing apparatus 100 b, thecommunication control unit 11 receives the transmission data 23D₁ andsends the transmission data 23D₁ to the drawing unit 24 (step S107). Thedrawing unit 24 draws the page object of a page corresponding to thepage number in the transmission data 23D₁ on the screen and therebyswitches pages (step S108). Thus, at the receiving informationprocessing apparatus 100 b, the page turning operation performed at thetransmitting information processing apparatus 100 a is reproduced on thescreen according to the page number in the received transmission data23D₁.

(2) Sharing Screen Reflecting Pointer Operation

As illustrated in FIG. 5, when a pointer operation is performed on thescreen of the transmitting information processing apparatus 100 a, acorresponding operation event is input to the drawing unit 24 and theoperation type determining unit 22.

According to the operation event, the drawing unit 24 draws a pointerobject on the screen (step S201).

Meanwhile, the operation type determining unit determines that theperformed input operation is a pointer operation based on the operationevent (step S202), and requests the transmission data generating unit togenerate transmission data 23D₂ (step S203). In this step, the operationtype determining unit 22 obtains coordinates indicating a pointerposition on the screen from the operation event and sends thecoordinates as operational information to the transmission datagenerating unit 23.

The transmission data generating unit 23 generates the transmission data23D₂ including the coordinates (step S204), and requests thecommunication control unit 11 to send the transmission data 23D₂ (stepS205). In this step, the transmission data generating unit 23 sends thegenerated transmission data 23D₂ to the communication control unit 11.

When the size of the screen of the receiving information processingapparatus 100 b is different from that of the transmitting informationprocessing apparatus 100 a, the transmission data 23D₂ may also includeinformation for adjusting the pointer position. For example, thetransmission data 23D₂ may include, in addition to the coordinates,information (screen frame information) indicating the screen frame sizeof the transmitting information processing apparatus 100 a.

Then, the communication control unit 11 sends the transmission data 23D₂including the coordinates indicating the pointer position and the screenframe size to the receiving information processing apparatus 100 b (stepS206).

At the receiving information processing apparatus 100 b, thecommunication control unit 11 receives the transmission data 23D₂ andsends the transmission data 23D₂ to the drawing unit 24 (step S207). Thedrawing unit 24 draws a pointer object on the screen according to thecoordinates and the screen frame size in the transmission data 23D₂(step S208). In this step, the drawing unit 24 adjusts the drawingposition of the pointer object by converting the coordinate space of thescreen of the transmitting information processing apparatus 100 a intothe coordinate space of the receiving information processing apparatus100 b (i.e., performing coordinate transformation) based on the screenframe size in the transmission data 23D₂ and the screen frame size ofthe receiving information processing apparatus 100 b.

Thus, at the receiving information processing apparatus 100 b, a pointeroperation performed at the transmitting information processing apparatus100 a is reproduced on the screen according to the coordinates and thescreen frame size in the received transmission data 23D₂.

<Summary>

As described above, in the screen sharing system 1000 of thisembodiment, the same content is displayed on the screens of informationprocessing apparatuses. At the information processing apparatus(transmitting information processing apparatus) 100 a having theoperation right, the operation type determining unit 22 determines aperformed input operation, the transmission data generating unit 23generates transmission data 23D including operational informationcorresponding to the determined input operation, and the communicationcontrol unit 11 transmits the transmission data 23D to the informationprocessing apparatus (receiving information processing apparatus) 100 b.At the receiving information processing apparatus 100 b, thecommunication control unit 11 receives the transmission data 23D, andthe drawing unit 100 reproduces the input operation on the screenaccording to the operational information in the received transmissiondata 23D.

Thus, the screen sharing system 1000 of this embodiment makes itpossible to share a display screen reflecting an input operation byexchanging a small amount of data among information processingapparatuses. This in turn makes it possible to reduce the load ofcommunications (or communication time) among the information processingapparatuses and to reduce the processing load (or screen drawing time)of the information processing apparatuses. Accordingly, the screensharing system 1000 of this embodiment makes it possible to achievereal-time screen sharing.

<First Variation>

A first variation of the above embodiment is described below.

FIG. 6 is a drawing illustrating an exemplary screen of a presenter andan exemplary screen of an audience. In FIG. 6, W1 indicated a displayscreen of a presenter and W2 indicates a display screen of an audiencein a situation as illustrated in FIG. 6.

In the example of FIG. 6, the presenter (a participant having theoperation right) moves a pointer object displayed on the display screenW1 from a position P1 to a position P2 during a presentation. Toreproduce the pointer operation by the presenter on the display screenW2 of the audience, multiple sets of coordinates indicating pointerpositions are sent and received periodically between the informationprocessing apparatus of the presenter (transmitting informationprocessing apparatus) and the information processing apparatus of theaudience (receiving information processing apparatus) while thepresenter is moving the pointer. In other words, transmission data 23D₂including the coordinates are sent and received between the informationprocessing apparatuses multiple times.

In the first variation, to reduce the communication frequency andthereby reduce the communication load, transmission data includingoperational information are sent to the receiving information processingapparatus at once (at one time) for each pointer moving operation. Alsoin the first variation, even when operational information is sent atonce for each input operation, a time lag between the actual inputoperation and the reproduction of the input operation at the receivinginformation processing apparatus is reduced or eliminated to achievereal-time screen sharing.

<Process of Sharing Screen Reflecting Pointer Moving Operation>

FIG. 7 is a sequence chart illustrating an exemplary process of sharinga screen when a pointer moving operation is performed.

As illustrated in FIG. 7, when a pointer moving operation is started onthe screen of the transmitting information processing apparatus 100 a, acorresponding operation event is input to the drawing unit 24 and theoperation type determining unit 22.

According to the operation event, the drawing unit 24 starts drawing apointer object on the screen (step S301).

Meanwhile, the operation type determining unit 22 determines that theperformed input operation is a pointer moving operation based on theoperation event (step S302), and sends an operation start reportindicating that the pointer moving operation has been started to thetransmission data generating unit 23 (step S303). In this step, theoperation type determining unit obtains starting coordinates indicatinga starting pointer position on the screen from the operation event andsends the starting coordinates as operational information to thetransmission data generating unit 23.

When receiving the operation start report (a request to generatetransmission data), the transmission data generating unit 23 starts aprocess as described below. The transmission data generating unit 23starts counting operation time T1 and recording coordinates indicatingpointer positions of the pointer being moved (step S304). For example,the transmission data generating unit 23 counts the operation time T1using a clock circuit (not shown) of the transmitting informationprocessing apparatus 100 a. Also, the transmission data generating unit23 buffers coordinates of pointer positions that are output periodicallyfrom the operation type determining unit 22 in a memory (e.g., the RAM104). Accordingly, after a certain period of time, multiple sets ofcoordinates indicating a trace of the pointer being moved aretemporarily stored in the memory.

When the pointer moving operation is completed on the screen of thetransmitting information processing apparatus 100 a, a correspondingoperation event is input to the drawing unit 24 and the operation typedetermining unit 22. For example, when the pointer moving operation isperformed by touching the screen, the completion of the pointer movingoperation is detected by the input unit 101 based on whether a pointingpart such as a touch pen or a finger is in contact with the screen. Theinput unit 101 determines that the pointer moving operation is beingperformed when the pointing part is in contact with the screen anddetermines that the pointer moving operation is completed when thepointing part is lifted from the screen.

According to the operation event (or the detection result), the drawingunit 24 stops drawing the pointer object on the screen (step S401).

Meanwhile, based on the operation event (or the detection result), theoperation type determining unit 22 sends an operation end reportindicating that the pointer moving operation has been completed to thetransmission data generating unit 23 (step S402). In this step, theoperation type determining unit 22 obtains ending coordinates indicatingan ending pointer position on the screen from the operation event, sendsthe ending coordinates as operational information to the transmissiondata generating unit 23, and thereby requests the transmission datagenerating unit 23 to generate the transmission data 23D₂.

When receiving the operation end report (a request to generatetransmission data), the transmission data generating unit 23 stopscounting the operation time T1 and recording the coordinates, andgenerates transmission data 23D₂ as illustrated in FIG. 8 (step S403).

FIG. 8 is a drawing illustrating exemplary transmission data 23D₂.

As illustrated in FIG. 8, the transmission data 23D₂ includes sets ofrecorded coordinates (x=a₁, y=b₂ . . . x=a_(n), y=b_(n)), a screen framesize (fsize=W [width]×H [height]), and counted operation time (time=T1[sec]).

Referring back to FIG. 7, the transmission data generating unit 23outputs the generated transmission data 23D₂ to the communicationcontrol unit 11 and requests the communication control unit 11 to sendthe transmission data 23D₂ (step S404).

Then, the communication control unit 11 sends the transmission data 23D₂to the receiving information processing apparatus 100 b (step S405). Inother words, multiple sets of the coordinates (arranged in the orderrecorded) indicating pointer positions from the start to the end of thepointer moving operation, the screen frame size of the transmittinginformation processing apparatus 100 a, and the operation time T1 takenfor the pointer moving operation are sent to the receiving informationprocessing apparatus 100 b at once (at one time). Thus, the firstvariation makes it possible to reduce the frequency of communicationsamong the information processing apparatuses and thereby reduce thecommunication load.

At the receiving information processing apparatus 100 b, thecommunication control unit 11 receives the transmission data 23D₂ andsends the transmission data 23D₂ to the drawing unit 24 (step S406).

The drawing unit 24 calculates time (pointer object drawing time T2) forreproducing the pointer moving operation at the receiving informationprocessing apparatus 100 b (step S407). The drawing unit 24 calculatesthe pointer object drawing time T2 using formula 1 below:Drawing time T2=operation time T1×adjustment factor k  (formula 1)Adjustment factor k: 0<k<1  (formula 2)

As is apparent from formula 1, the adjustment factor k is used to adjustthe speed (drawing speed) of reproducing the input operation at thereceiving information processing apparatus 100 b.

An exemplary relationship between the operation time T1 and theadjustment factor k is described below.

FIG. 9 is a graph illustrating an exemplary relationship between theoperation time T1 and the adjustment factor k. In FIG. 9, the verticalaxis indicates the adjustment factor k and the horizontal axis indicatesthe operation time T1.

As illustrated in FIG. 9, the operation time T1 and the adjustmentfactor k are inversely proportional to each other. The adjustment factork is within a range indicated by formula 2 above. When the operationtime T1 is long, the adjustment factor k is set at a decimal fractionclose to “0”. Meanwhile, when the operation time T1 is short, theadjustment factor k is set at a decimal fraction close to “1”.

Based on the above relationship and the operation time T1, the drawingunit 24 determines the adjustment factor k and calculates the drawingtime T2. For example, the drawing unit 24 accesses a storage areastoring data (e.g., a table) indicating the correspondence betweenoperation time T1 and adjustment factors k, and obtains the adjustmentfactor k corresponding to the operation time T1. Then, the drawing unit24 sets the obtained adjustment factor k and the operation time T1 inthe transmission data 23D₂, and thereby calculates the drawing time T2.

Thus, the drawing unit 24 determines the adjustment factor k based onthe operation time T1 and adjusts the speed (drawing speed) ofreproducing the input operation at the receiving information processingapparatus 100 b based on the determined adjustment factor k.

FIG. 10 is a drawing used to describe an exemplary process of sharing ascreen where a pointer is moved.

In the first variation, for example, when the operation time T1 of aninput operation performed at the transmitting information processingapparatus 100 a is long, drawing time T2 that is less than the operationtime T1 is calculated using an adjustment factor k that is a decimalfraction close to “0” so that the drawing speed is increased.

This configuration makes it possible to reduce or eliminate the time lagbetween the actual input operation at the transmitting informationprocessing apparatus and the reproduction of the input operation at thereceiving information processing apparatus, and thereby makes itpossible to achieve real-time screen sharing.

Referring back to FIG. 7, after calculating the drawing time T2, thedrawing unit 24 starts drawing the pointer object on the screenaccording to the sets of coordinates (arranged in the order recorded)and the screen frame size included in the transmission data 23D₂ (stepS408). The drawing unit 24 draws the pointer object at such a speed thatthe drawing process is completed within the calculated drawing time T2.As a result, the pointer moving operation is reproduced at a speedhigher than the speed at which the pointer moving operation has beenperformed at the transmitting information processing apparatus 100 a.

Thus, in the screen sharing system 1000 of the first variation,transmission data including operational information are sent to thereceiving information processing apparatus at once (at one time) foreach input operation to reduce the communication frequency and thecommunication load. Also in the screen sharing system 1000 of the firstvariation, the speed of reproducing an input operation at the receivinginformation processing apparatus is adjusted to reduce or eliminate thetime lag caused by sending the operational information at once andthereby achieve real-time screen sharing.

In the first variation, a pointer moving operation is used as anexample. However, the first variation may also be applied to other typesof input operations such as a handwriting input operation. In otherwords, the first variation may be applied to an input operationperformed by continuously touching the screen for a certain period oftime.

<Second Variation>

A second variation of the above embodiment is described below.

In the second variation, it is assumed that an audience records apersonal memo on a presentation material being shared by participants ofa presentation meeting while a presenter (a participant having theoperation right) is performing a presentation.

For this purpose, the second variation provides a mechanism forswitching shared and non-shared modes (screen sharing modes) of adisplay screen.

FIG. 11 is a block diagram illustrating an exemplary functionalconfiguration of the information processing apparatus 100 according tothe second variation.

The functional configuration of FIG. 11 is different from the functionalconfiguration of FIG. 3 in that a sharing mode switching unit 25 isadded.

The sharing mode switching unit 25 is a functional unit that switchesshared and non-shared modes of a display screen. The sharing modeswitching unit 25 displays a graphical user interface (GUI) forreceiving a sharing mode switching request (a shared mode request or anon-shared mode request), and switches screen sharing modes according tothe received sharing mode switching request.

In the descriptions below, it is assumed that screen sharing modes areswitched at an information processing apparatus of an audience, i.e.,the receiving information processing apparatus 100 b.

When the non-shared mode request is received, the sharing mode switchingunit 25 enables the function for receiving input operations on theinformation processing apparatus 100 b and activates the operation typedetermining unit 22. At the same time, the sharing mode switching unit25 requests the drawing unit 24 to switch the display screen from ashared screen to a personal screen. The operation type determining unit22 receives an input operation performed on the personal screen andrecords operational information of the input operation in an operationalinformation storing unit 31. The operational information storing unit 31is, for example, implemented by a storage area of a storage unit (e.g.,the HDD 108) of the information processing apparatus 100 b. Thisconfiguration enables the audience to record a personal memo byperforming, for example, a handwriting input operation on theinformation processing apparatus 100 b.

Also, when receiving the non-shared mode request, the sharing modeswitching unit 25 requests the communication control unit 11 to recordtransmission data 23D (operational information) received from thetransmitting information processing apparatus 100 a. The communicationcontrol unit 11 receives the transmission data 23D from the transmittinginformation processing apparatus 100 a and records the receivedtransmission data 23D in a received data storing unit 32. The receiveddata storing unit 32 is, for example, implemented by a storage area of astorage unit (e.g., the HDD 108) of the information processing apparatus100 b. The storage area used for the received data storing unit 32 isdifferent from the storage area used for the operational informationstoring unit 31. Thus, in the second variation, input operations on thepersonal screen and input operations on the shared screen are stored indifferent storage areas.

When the screen sharing mode is switched from the non-sharing mode tothe sharing mode, the operation type determining unit 22 stops recordingoperational information in the operational information storing unit andis deactivated. Meanwhile, the communication control unit 11 stopsrecording transmission data in the received data storing unit 32, andthe drawing unit 24 switches screens from the personal screen to theshared screen and reproduces input operations performed at thetransmitting information processing apparatus 100 a based on thetransmission data stored in the received data storing unit 32.

Thus, the screen sharing function of the second variation is implementedthrough collaboration among the functional units of the informationprocessing apparatuses 100.

<Process of Switching Screen Sharing Modes>

FIG. 12 is a sequence chart illustrating an exemplary process ofswitching screen sharing modes according to the second variation.

As illustrated in FIG. 12, when the non-shared mode request is enteredat the receiving information processing apparatus 100 b, the sharingmode switching unit 25 switches the screen sharing mode from the sharedmode to the non-shared mode, and requests the communication control unit11 to start recording transmission data 23D received from thetransmitting information processing apparatus 100 a (step S501).

The communication control unit 11 receives the transmission data 23Dfrom the transmitting information processing apparatus 100 a and recordsthe received transmission data 23D in the received data storing unit 32(step S502).

Also, the sharing mode switching unit 25 reports to the drawing unit 24and the operation type determining unit 22 that the screen sharing modehas been switched from the shared mode to the non-shared mode (stepS503).

In response, the drawing unit 24 switches the display screen from theshared screen to the personal screen (step S504).

When a handwriting input operation is performed on the personal screen,a corresponding operation event is input to the drawing unit 24 and theoperation type determining unit 22.

According to the operation event, the drawing unit 24 draws a lineobject corresponding to the trace of handwriting on the personal screen(step S601).

Meanwhile, the operation type determining unit 22 determines that theperformed input operation is a handwriting input operation based on theoperation event (step S602), and records the operational information ofthe handwriting input operation in the operational information storingunit 31 (step S603). For example, the operation type determining unit 22records operational information including the page number of a page onwhich the handwriting input operation has been performed and multiplesets of coordinates indicating the trace of handwriting (i.e., sets ofcoordinates forming a line).

When the shared mode request is entered at the receiving informationprocessing apparatus 100 b, the sharing mode switching unit 25 switchesthe screen sharing mode from the non-shared mode to the shared mode, andrequests the communication control unit 11 to stop recording thetransmission data 23D received from the transmitting informationprocessing apparatus 100 a (step S701). Then, the communication controlunit 11 stops recording the transmission data 23D in the received datastoring unit 32.

Also, the sharing mode switching unit 25 reports to the drawing unit 24and the operation type determining unit 22 that the screen sharing modehas been switched from the non-shared mode to the shared mode (stepS702). In response, the operation type determining unit 22 stopsrecording operational information in the operational information storingunit 31.

Meanwhile, the drawing unit 24 switches the display screen from thepersonal screen to the shared screen (step S703), accesses the receiveddata storing unit 32, and reproduces input operations performed at thetransmitting information processing apparatus 100 a on the shared screenaccording to the operational information in the transmission data 23Dstored in the received data storing unit 32 (step S704).

Thus, the screen sharing system 1000 of the second variation makes itpossible to improve the convenience of the screen sharing service.

To achieve real-time screen sharing, the method or configuration of thefirst variation for adjusting the drawing speed at the receivinginformation processing apparatus may also be applied to a process in thesecond variation of reproducing input operations on the shared screenafter the screen sharing mode is switched from the non-sharing mode tothe sharing mode.

<Third Variation>

A third variation of the above embodiment is described below.

FIGS. 13A and 13B are drawings illustrating an exemplary configurationof a screen sharing system 1010 according to the third variation.

The configuration of FIG. 13A is different from the configuration ofFIG. 1A in that a projection apparatus 200 is connected to the datacommunication channel N.

The projection apparatus 200 may be a projector that includes a radiocommunication function and an information display function.

FIG. 13B illustrates an exemplary case where the screen sharing system1010 is used for a presentation meeting. In this exemplary case, theprojection apparatus 200 shares a display screen with the informationprocessing apparatus 100 ₁ of the presenter in a manner similar to theinformation processing apparatuses 100 ₂ and 100 ₃ of the audiences,except that the projection apparatus 200 projects the shared screen ontoa projection screen.

Similarly to the information processing apparatus 100 of the aboveembodiment, the projection apparatus 200 includes hardware componentsfor implementing the screen sharing function.

FIG. 14 is a block diagram illustrating an exemplary hardwareconfiguration of the projection apparatus 200 according to the thirdvariation.

As illustrated in FIG. 14, the projection apparatus 200 includes acontroller 210, an operations panel 220, and a projecting unit 230 thatare connected to each other via a bus B.

The operations panel 220 includes a display unit for providinginformation such as device information to the user and an input unit forreceiving user inputs such as settings and instructions. The projectingunit 230 projects image data onto a projection screen.

The controller 210 is a control circuit board including a CPU 211, astorage unit 212, a network I/F 213, and an external storage I/F 214that are connected via the bus B.

The storage unit 211 includes a RAM, a ROM, and an HDD for storingprograms and data. The CPU 212 loads programs and data from the ROM andthe HDD into the RAM and executes the loaded programs to control theprojection apparatus 200 and thereby implement various functions.

The network I/F 213 is an interface for connecting the projectionapparatus 200 to the data communication channel N. The projectionapparatus 200 can perform data communications with the informationprocessing apparatuses 100 via the network I/F 213.

The external storage I/F 214 is an interface for connecting a storagemedium 214 a used as an external storage to the projection apparatus200. Examples of the storage medium 214 a include an SD memory card, aUSB memory, a CD, and a DVD. The projection apparatus 200 can read andwrite data from and to the storage medium 214 a via the external storageI/F 214.

In the projection apparatus 200 with the hardware configuration asdescribed above, the CPU 211 loads a screen sharing application program(software for implementing the screen sharing function) from the HDD orthe ROM into the RAM and executes the loaded screen sharing applicationprogram to implement the screen sharing function.

The screen sharing system 1010 of the third variation also providesadvantageous effects similar to those provided by the screen sharingsystem 1000 of the above embodiment.

The screen sharing function of the above embodiment and variations maybe implemented by executing programs by processing units (e.g., CPUs) ofapparatuses (i.e., the information processing apparatuses 100 and theprojection apparatus 200) constituting the screen sharing system1000/1010. The programs may be written in a programming language(s)supported by the operating environments (platforms) of the apparatuses.

The programs to be executed by the information processing apparatus 100and the projection apparatus 200 may be stored in non-transitorycomputer-readable storage media (e.g., the storage medium 103 a and thestorage medium 214 a), and may be installed into the informationprocessing apparatus 100 via the drive unit 103 and into the projectionapparatus 200 via the external storage I/F 214. Alternatively, theprograms may be installed via a telecommunication line and the interfaceunit 107 into the information processing apparatus 100, and may beinstalled via a telecommunication line and the network I/F 213 into theprojection apparatus 200.

An aspect of this disclosure provides a screen sharing system, a screensharing method, and a storage medium storing a program that make itpossible to share a display screen reflecting an input operation byexchanging a small amount of data.

The present invention is not limited to the specifically disclosedembodiments, and variations and modifications may be made withoutdeparting from the scope of the present invention.

What is claimed is:
 1. A system, comprising: a first informationprocessing apparatus including a first display; and a second informationprocessing apparatus that includes a second display and is connectableto the first information processing apparatus via a network, wherein thefirst information processing apparatus includes a memory storing aprogram and a processor configured to execute the program to perform aprocess including storing, in the memory, a display content thatincludes multiple pages and is shared with the second informationprocessing apparatus, displaying the display content stored in thememory on the first display, displaying, on the first display, a firstgraphical user interface for receiving a right request to obtain a rightfor performing one or more input operations including at least a pageturning operation on the display content, displaying, on the firstdisplay, a page of the display content that is same as a page displayedon the second display of the second information processing apparatus inresponse to a page turning operation performed on the second informationprocessing apparatus by a presenter having the right, displaying, on thefirst display, a second graphical user interface for receiving requeststo switch a screen-sharing mode between a shared mode and a non-sharedmode, when the screen-sharing mode is switched from the shared mode tothe non-shared mode according to a request received by the secondgraphical user interface, enabling the first information processingapparatus to independently receive the page turning operation on thedisplay content, and when the screen-sharing mode is switched from thenon-shared mode to the shared mode according to a request received bythe second graphical user interface while the first display isdisplaying a page of the display content different from the page beingdisplayed on the second display of the second information processingapparatus operated by the presenter having the right, causing the firstdisplay to display a page of the display content that is the same as thepage being displayed on the second display, according to operationinformation sent from the second information processing apparatus andindicating the page turning operation performed by the presenter.
 2. Thesystem as claimed in claim 1, wherein the one or more input operationsthat have become receivable at the first information processingapparatus switched to the non-shared mode are not reproduced on thesecond information processing apparatus.
 3. The system as claimed inclaim 1, wherein in response to the right request to obtain the right,the first information processing apparatus switches an input mode froman input operation disabled mode where the one or more input operationsare not accepted to an input operation enabled mode where the one ormore input operations are accepted; and the first information processingapparatus receiving the right request obtains the right when the secondinformation processing apparatus having the right switches the inputmode from the input operation enabled mode to the input operationdisabled mode.
 4. The system as claimed in claim 1, wherein when a rightmode of the first information processing apparatus is switched from ano-right mode not having the right to a right-given mode having theright in response to the right request to obtain the right, the firstinformation processing apparatus enables a function for receiving theone or more input operations on the first display and sends a report, tothe second information processing apparatus, to disable a function forreceiving the one or more input operations on the second display of thesecond information processing apparatus; and in response to the reportfrom the first information processing apparatus, the second informationprocessing apparatus previously having the right disables the functionfor receiving the one or more input operations on the second display. 5.The system as claimed in claim 1, wherein the process further includesobtaining the display content from the memory of the first informationprocessing apparatus.
 6. The system as claimed in claim 1, wherein whena pointer operation by a user is received, the first informationprocessing apparatus sends pointer-position information indicating aposition of a pointer on the first display.
 7. The system as claimed inclaim 1, wherein the one or more input operations restricted by theright further include at least one of a pointer operation and ahandwriting input operation on the first display.
 8. The system asclaimed in claim 1, wherein each of the first information processingapparatus and the second information processing apparatus includes ascreen sharing application installed therein; and each of the firstinformation processing apparatus and the second information processingapparatus starts the screen sharing application to display the displaycontent on a corresponding one of the first and second displays.
 9. Thesystem as claimed in claim 1, wherein each of the first informationprocessing apparatus and the second information processing apparatus isone of a smartphone and a tablet personal computer.
 10. An informationprocessing apparatus for a system including the information processingapparatus and another information processing apparatus that areconnectable to each other via a network, the information processingapparatus comprising: a first display; a memory storing a program; and aprocessor configured to execute the program to perform a processincluding storing, in the memory, a display content that includesmultiple pages and is shared with the another information processingapparatus, displaying the display content stored in the memory on thefirst display, displaying, on the first display, a first graphical userinterface for receiving a right request to obtain a right for performingone or more input operations including at least a page turning operationon the display content, displaying, on the first display, a page of thedisplay content that is same as a page displayed on a second display ofthe another information processing apparatus in response to a pageturning operation performed on the another information processingapparatus by a presenter having the right, displaying, on the firstdisplay, a second graphical user interface for receiving requests toswitch a screen-sharing mode between a shared mode and a non-sharedmode, when the screen-sharing mode is switched from the shared mode tothe non-shared mode according to a request received by the secondgraphical user interface, enabling the information processing apparatusto independently receive the page turning operation on the displaycontent, and when the screen-sharing mode is switched from thenon-shared mode to the shared mode according to a request received bythe second graphical user interface while the first display isdisplaying a page of the display content different from the page beingdisplayed on the second display of the another information processingapparatus operated by the presenter having the right, causing the firstdisplay to display a page of the display content that is the same as thepage being displayed on the second display, according to operationinformation sent from the another information processing apparatus andindicating the page turning operation performed by the presenter.
 11. Amethod performed by an information processing apparatus for a systemincluding the information processing apparatus and another informationprocessing apparatus that are connectable to each other via a network,the method comprising: storing, in a memory of the informationprocessing apparatus, a display content that includes multiple pages andis shared with the another information processing apparatus; displayingthe display content stored in the memory on a first display of theinformation processing apparatus; displaying, on the first display, afirst graphical user interface for receiving a right request to obtain aright for performing one or more input operations including at least apage turning operation on the display content; displaying, on the firstdisplay, a page of the display content that is same as a page displayedon a second display of the another information processing apparatus inresponse to a page turning operation performed on the anotherinformation processing apparatus by a presenter having the right;displaying, on the first display, a second graphical user interface forreceiving requests to switch a screen-sharing mode between a shared modeand a non-shared mode; when the screen-sharing mode is switched from theshared mode to the non-shared mode according to a request received bythe second graphical user interface, enabling the information processingapparatus to independently receive the page turning operation on thedisplay content; and when the screen-sharing mode is switched from thenon-shared mode to the shared mode according to a request received bythe second graphical user interface while the first display isdisplaying a page of the display content different from the page beingdisplayed on the second display of the another information processingapparatus operated by the presenter having the right, causing the firstdisplay to display a page of the display content that is the same as thepage being displayed on the second display, according to operationinformation sent from the another information processing apparatus andindicating the page turning operation performed by the presenter.